What Is Virtual Reality? 


1.1 The Definition of Virtual Reality 


The term virtual reality (VR) 1s commonly used by the popular media to describe imaginary worlds that 
only exist in computers and our minds. However, let us more precisely define the term. Sherman and 
Craig [2003] poimt out in their book Understanding Virtual Reality that Webster's New Universal 
Unabridged Dictionary [1989] defimes virtual as “being mm essence or effect, but not in fact” and reality 
as “the state or quality of being real. Something that exists independently of ideas concerning 1t. 
Something that constitutes a real or actual thing as distinguished from something that is merely apparent.” 
Thus, virtual reality 1s a term that contradicts itself-—an oxymoron! Fortunately, the website merriam- 
webster.com [Merriam-Webster 2015] has more recently defined the full term virtual reality to be “an 
artificial environment which 1s experienced through sensory stimuli (as sights and sounds) provided by 
a computer and in which one's actions partially determine what happens in the environment.” In this 
book, virtual reality 1s defined to be a computer-generated digital environment that can be experienced 
and interacted with as 1f that environment were real. 


An ideal VR system enables users to physically walk around objects and touch those objects as 1 
they were real. Ivan Sutherland, the creator of one of the world”s first VR systems 1n the 1960s, stated 
[Sutherland 1965]: “The ultimate display would, of course, be a room within which the computer can 
control the existence of matter. A chair displayed m such a room would be good enough to sit in. 
Handcuffs displayed in such a room would be confinimg, and a bullet displayed m such a room would 
be fatal.” We haven't yet come anywhere near Ivan Sutherland”s vision (nor do we necessarily want 
to!) and perhaps we never will. However, there are some quite engaging virtual realities today — 
many of which are featured throughout this book. 


1.2 VR Is Communication 


Normally, communication 1s thought of as interaction between two or more people. This book defines 
communication more abstractly: the transfer of energy between two entities, even 1f just the cause and 
effect of one object colliding with another object. Communication can also be between human and 
technology —an essential component and basis of VR. VR design 1s concerned with the communication 
of how the virtual world works, how that world and its objects are controlled, and the relationship 
between user and content: ideally where users are focused on the experience rather than the technology. 


Well-designed VR experiences can be thought of as collaboration between human and machine 
where both software and hardware work harmoniously together to provide intuitive communication with 
the human. Developers write complex software to create, 1f designed well, seemingly simple transfer 
functions to provide effective interactions and engaging experiences. Communication can be 


broken down into direct communication and indirect communication as discussed below. 


1.2.1 Direct Communication 


Direct communication is the direct transfer of energy between two entities with no intermediary and 
no interpretation attached. In the real world, pure direct communication between entities doesn”t 
represent anything as the purpose is not communication, but it is a side effect. However, in VR, 
developers insert an artificial intermediary (the VR system that 1s 1deally unperceivable) between the 
user and carefully controlled sensory stimuli (e.g., shapes, motions, sounds). When the goal is direct 
communication, VR creators should focus on making the intermediary transparent so users feel like they 
have direct access to those entities. If that can be achieved, then users will perceive, interpret, and 
interact with stimuli as 1f they are directly communicating with the virtual world and 1ts entities. 


Direct communication consists of structural communication and visceral communication. 


Structural Communication 


Structural communication is the physics of the world, not the description or the mathematical 
representation but the thing-in-itself [Kant 1781]. An example of structural communication 1s the 
bouncing of a ball off of the hand. We are always 1n relationship to objects, which help to define our 
state; e.g., the shape of our hand around a controller. The world, as well as our own bodies, directly 
tells us what the structure 1s through our senses. Although thinking and feeling do not exist within 
structural communication, such communication does provide the starting point for perception, 
interpretation, thinking, and feeling. 

In order for our ideas to persist through time, we must put those ideas into structural form, what 
Norman [2013] calls knowledge in the world. Recorded information and data is the obvious example 
of structural form, but sometimes less obvious structural forms are the signifiers and constraints (Section 
25.1) of interaction. In order to induce experiences into others through VR, we presen structural stimuli 
(e.g., pixels on a display, sound through headphones, or the rumble/vibration of a controller) so the 
users can sense and interact with our creations. 


Visceral Communication 


Visceral communication is the language of automatic emotion and primal behavior, not the rational 
representation of the emotions and behavior (Section 7.7). Visceral communication 1s always present 
for humans and is the in-between of structural communication and indirect communication. Presence 
(Chapter 4) 1s the act of being fully engaged via direct communication (albeit primarily one way). 
Examples of visceral communication are the feeling of awe while sitting on a mountaintop, looking 
down at the earth from space, or being with someone via solid eye contact (whether in the real world 
or via avatars mn VR). The full experience of such visceral communication cannot be put into words 
although we often attempt to do so at which point the experience 1s transformed into indirect 
communication (e.g., explaining VR to someone 1s not the same as experiencing VR). 


1.2.2 Indirect Communication 


Indirect communication connects two or more entities through some intermediary. The intermediary 
need not be physical; in fact, the intermediary 1s often our mind”s interpretation that sits between the 


world and behavior/action. Once we interpret and give something meaning, then we have transformed 
the direct communication into indirect communication. Indirect communication includes what we 
normally think of as language, such as spoken and written language, as well as sign languages and our 
internal thoughts (1.e., communicating with oneself). Indirect communication consists of talking, 
understanding, creating stories/histories, giving meaning, comparing, negating, fantasizing, lying, and 
romancing. These are not part of the objective real world but are what we humans describe and 
create with our minds. Indirect VR communication includes the user”s internal mental model (Sectio 
7.8) of how the VR world works (e.g., the interpretation of what 1s occurring in the VR world), and 
indirect interactions (Section 28.4) such as moving a slider that changes an object property, speech 
recognition that changes the system”s state, and indirect gestures that act as a sign language to the 
computer. 


1.3 What Is VR Good For? 


The recent surge in media coverage about VR has inspired the public to become quite excited abou 1ts 
potential. This coverage has focused on the entertaanment industry, specifically video games and 
immersive film. VR 1s a great fit for the entertainment industry and will certainly be the driving force 
behind VR in the short term. However, what 1s VR good for beyond entertannment? It turns out VR ca 
have enormous benefit over a wide range of verticals. VR has been successfully deployed in various 
industries for many years now. Successful applications include oil and gas exploration, scientific 
visualization, architecture, flight simulation, therapy, military training, theme-park entertainment, 
engineering analysis, and design review. Using VR im such situations has successfully revealed costly 
design mistakes before manufacturing anything, reduced time to market by speeding up iterative 
processes, provided safe learning environments that would otherwise be dangerous, reduced PTSD by 
gradually increasing exposure to feared stimul, and helped to visualize large datasets that would be 
difficult to comprehend with traditional systems. 


Unfortunately, to date, VR has largely been limited to well-funded academic and corporate research 
labs to which few have access. That is all changing with consumer-priced systems now becoming widely 
available. The VR market is expected to first explode 1n the entertainment industry but will soon expand 
significantly in other industries. Education, telepresence, and professional training will likely be the 
next industries to take advantage of VRin a big way. 


Whatever the industry, VR 1s largely about providing understanding —whether that 1s understanding 
an entertaining story, learning an abstract concept, or practicing a real skill. Actively using more of the 
human sensory capability and motor skills has been known to increase understanding/learning for some 
time [Dale 1969]. This is m part due to the increased sensory bandwidth between human and 
information, but there is much more to understanding. Actively participating im an action, making 
concepts intuitive, encouraging motivation through engaging experiences, and the thoughts inside one”s 
head all contribute to understanding. This book focuses on how to design such concepts into VR 
experiences. 

Figure 1.1 shows Edgar Dale”s Cone of Experience [Dale 1969]. As can be seen in the figure, direct 
purposeful experiences provide the best basis for understanding. As Confucius stated, “I see and I forget. 
I hear and I remember. I do and I understand.” Note this diagram does not suggest direc purposeful 
experiences should be the only method of learning, but instead describes the progression 


of learning experience. Adding other indirect information within direct purposeful VR experiences can 
further enhance understanding. For example, embedding abstract information such as text, symbols, and 
multimedia directly into the scene and onto virtual objects can lead to more efficient understanding than 
what can be achieved in the real world. 
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Figure 1.1 The Cone of Expe rie nce . VR use s many le ve Is of abstraction. (Adapted from Dale [1969]) 


